library(tidyverse)
library(datasauRus)
library(GGally)
library(reshape)

Relação entre variáveis

Em análise exploratória, com frequência queremos examinar se há associação entre duas variáveis numéricas. Para descrever uma associação como essa, há pelo menos quatro características importantes:

  1. Formato da associação: linear, exponencial, parabólica, linear e depois assintótica, outro formato arbitrário, etc.
  2. Força da associação: correlação forte, fraca, nenhuma.
  3. Sinal da associação: correlação positiva ou negativa, quando é perceptível.
  4. Pontos extremos fora da associação.

Para relacionar variáveis, podemos usar técnicas de visualização como gráficos de dispersão (scatter plots). Outra técnica é a análise de correlação.

Correlação

Correlação é uma medida que indica se a relação entre 2 variáveis é estatisticamente significativa.

O coeficiente de correlação de Pearson é o mais popular e usado principalmente para relações lineares. Outros coeficientes de correlação robustos e úteis para outros tipos de relação são os coeficientes de Spearman e Kendall.

Coeficiente de Correlação de Pearson

O coeficiente de correlação de Pearson indica a força e a direção da correlação linear entre duas variáveis. Seu coeficiente amostral para as variáveis x e y é definido por:

\[r_{xy} = \frac{1}{n-1} \sum_{i=1}^{n} \left( \frac{x_i - \bar{x}}{s_x} \right) \left( \frac{y_i - \bar{y}}{s_y} \right) \]

O valor do coeficiente \(r_{xy}\) varia de -1 a 1:

  • \(+1\): Relação positiva perfeita: quanto x aumenta, y aumenta linearmente.
  • \(-1\): Relação negativa perfeita: quanto x aumenta, y diminui linearmente.
  • \(0\) : Não há relação entre x e y

Tipicamente, se adota esses valores para indicar a força da correlação:

  • Se \(r_{xy} \gt 0,7\) ou \(r_{xy} \lt -0,7\): correlação forte
  • Se \(0,3 \lt r_{xy} \lt 0,7\) ou \(-0,7 \lt r_{xy} \lt - 0,3\): correlação moderada
  • Se \(-0,3 \lt r_{xy} \lt 0,3\): correlação fraca
  • Se \(r_{xy} \approx 0\): não há correlação (ou não se pode concluir nada)

Exemplo: análise de correlação de Pearson

Conselho importante: não há um número apenas que possa lhe responder tudo sobre a associação entre as duas variáveis. As medidas de correlação que você usará servirão mais para complementar e quantificar observações feitas em gráficos de dispersão do que para lhe guiar.

Um exemplo classico com quatro pares de variáveis cujas associações interessam: x1 e y1, x2 e y2, etc., o quarteto de Anscombe:

summary(anscombe)
       x1             x2             x3             x4           y1               y2       
 Min.   : 4.0   Min.   : 4.0   Min.   : 4.0   Min.   : 8   Min.   : 4.260   Min.   :3.100  
 1st Qu.: 6.5   1st Qu.: 6.5   1st Qu.: 6.5   1st Qu.: 8   1st Qu.: 6.315   1st Qu.:6.695  
 Median : 9.0   Median : 9.0   Median : 9.0   Median : 8   Median : 7.580   Median :8.140  
 Mean   : 9.0   Mean   : 9.0   Mean   : 9.0   Mean   : 9   Mean   : 7.501   Mean   :7.501  
 3rd Qu.:11.5   3rd Qu.:11.5   3rd Qu.:11.5   3rd Qu.: 8   3rd Qu.: 8.570   3rd Qu.:8.950  
 Max.   :14.0   Max.   :14.0   Max.   :14.0   Max.   :19   Max.   :10.840   Max.   :9.260  
       y3              y4        
 Min.   : 5.39   Min.   : 5.250  
 1st Qu.: 6.25   1st Qu.: 6.170  
 Median : 7.11   Median : 7.040  
 Mean   : 7.50   Mean   : 7.501  
 3rd Qu.: 7.98   3rd Qu.: 8.190  
 Max.   :12.74   Max.   :12.500  
# um pouco de rearrumação primeiro. 
# (para entender melhor, você pode procurar sobre tidy data em R)
t1 <- melt(select(anscombe, 1:4), id = c())
t2 <- melt(select(anscombe, 5:8), id = c())
dados <- data.frame(vars = paste(t1$variable, "e", t2$variable), 
                    v1 = t1$value, 
                    v2 = t2$value)

Calculando a correlação linear:

dados %>% 
  group_by(vars) %>% 
  summarise(correlacao_pearson = cor(v1, v2, method = "pearson"))

E se olharmos os dados?

ggplot(dados, aes(v1, v2)) + 
  geom_point(color = "darkorange", size = 4, alpha = 0.7) + 
  theme_bw() + 
  scale_x_continuous(breaks = seq(0, 20, 2)) + 
  scale_y_continuous(breaks = seq(0, 12, 2)) + 
  expand_limits(x = 0, y = 0) + 
  facet_wrap(~ vars)

Quatro relações diferentes, mesma quantificação. Para o segundo grupo, não há uma relação linear. No 3o, há uma relação perfeita entre a maioria das observações, com uma exceção. No 4o grupo não há relação; há uma exceção que faz parecer que há uma relação.

O que os outros coeficientes podem nos dizer?

dados %>% 
  group_by(vars) %>% 
  summarise(pearson = cor(v1, v2, method = "pearson"), 
            spearman = cor(v1, v2, method = "spearman"),
            kendall = cor(v1, v2, method = "kendall"))

Força e também direção

Dito isso, essa figura ajuda a lembrar a relação entre o valor esperado do coeficiente (linear) e vários tipos de associação entre duas variáveis:

da wikipedia

da wikipedia


Alguns testes gerando a relação

Relação linear:

set.seed(123)
x <- rnorm(100) * 100
tamanho_do_erro <- 50
y <- 0.5 * x + rnorm(100) * tamanho_do_erro + 20
df <- data.frame(x = x, 
                 y = y)
ggplot(df, aes(x = x, y = y)) + 
  geom_point(colour = "darkorange", size = 4) + 
  theme_bw()

cor(df$x, df$y, method = "pearson")
[1] 0.66713
# Dobro de erro em torno de uma função linear f(x)
df$y <- 0.5 * x + rnorm(100) * tamanho_do_erro * 2 + 20
ggplot(df, aes(x = x, y = y)) + 
  geom_point(colour = "darkorange", size = 4) + 
  theme_bw()

cor(df$x, df$y, method = "pearson")
[1] 0.3339445

Relação não linear

n = 100
df <- data.frame(x = runif(n, min= 1, max = 20))
df$y = 100 * exp(-1.2 * df$x) #+ rnorm(n, mean = 0.05, sd = 1)
ggplot(df, aes(x = x, y = y)) + 
  geom_point(colour = "darkorange", size = 4) +
  theme_bw()

  
cor(df$x, df$y)
[1] -0.5252222
cor(df$x, df$y, method = "spearman")
[1] -1
cor(df$x, df$y, method = "kendall")
[1] -1
# Em escala de log
ggplot(df, aes(x = x, y = y)) + 
  geom_point(colour = "darkorange", size = 4) +
  scale_y_log10() +
  theme_bw()


Correlação vs. Causalidade

ATENÇÃO! Correlação não implica causalidade. Alguns materiais sobre o tema:

Cuidado com sumários

datasaurus_dozen %>% 
  group_by(dataset) %>% 
  summarise_all(funs(mean, sd))

Correlações

datasaurus_dozen %>% 
  group_by(dataset) %>% 
  summarise(correlacao = cor(x, y))

Boxplots

datasaurus_dozen %>% 
  group_by(dataset) %>% 
  ggplot(aes(x = dataset, y = y)) + 
  geom_boxplot()

datasaurus_dozen %>% 
  group_by(dataset) %>% 
  ggplot(aes(x = dataset, y = y)) + 
  geom_violin()

ggplot(datasaurus_dozen, aes(x=x, y=y, colour=dataset))+
  geom_point()+
  theme(legend.position = "none")+
  facet_wrap(~dataset, ncol=4)

LS0tCnRpdGxlOiAiTWVkaWRhcyBkZSBjb3JyZWxhw6fDo28iCmF1dGhvcjogIk1hcmN1cyBDYXJ2YWxobyAoYWRhcHRhZG8gZGUgTmF6YXJlbm8gQW5kcmFkZSBlIFJhcXVlbCBMb3BlcykiCmRhdGU6ICJBbsOhbGlzZSBkZSBEYWRvcyBJIC0gMjAxOC4xIgpvdXRwdXQ6IGh0bWxfbm90ZWJvb2sKLS0tCgpgYGB7ciwgbWVzc2FnZT1GQUxTRSwgd2FybmluZz1GQUxTRX0KbGlicmFyeSh0aWR5dmVyc2UpCmxpYnJhcnkoZGF0YXNhdVJ1cykKbGlicmFyeShHR2FsbHkpCmxpYnJhcnkocmVzaGFwZSkKYGBgCgojIyBSZWxhw6fDo28gZW50cmUgdmFyacOhdmVpcwoKRW0gYW7DoWxpc2UgZXhwbG9yYXTDs3JpYSwgY29tIGZyZXF1w6puY2lhIHF1ZXJlbW9zIGV4YW1pbmFyIHNlIGjDoSBhc3NvY2lhw6fDo28gZW50cmUgZHVhcyB2YXJpw6F2ZWlzIG51bcOpcmljYXMuIFBhcmEgZGVzY3JldmVyIHVtYSBhc3NvY2lhw6fDo28gY29tbyBlc3NhLCBow6EgcGVsbyBtZW5vcyBxdWF0cm8gY2FyYWN0ZXLDrXN0aWNhcyBpbXBvcnRhbnRlczogCgogIDEuICpGb3JtYXRvKiBkYSBhc3NvY2lhw6fDo286IGxpbmVhciwgZXhwb25lbmNpYWwsIHBhcmFiw7NsaWNhLCBsaW5lYXIgZSBkZXBvaXMgYXNzaW50w7N0aWNhLCBvdXRybyBmb3JtYXRvIGFyYml0csOhcmlvLCBldGMuCiAgMi4gKkZvcsOnYSogZGEgYXNzb2NpYcOnw6NvOiBjb3JyZWxhw6fDo28gZm9ydGUsIGZyYWNhLCBuZW5odW1hLiAKICAzLiAqU2luYWwqIGRhIGFzc29jaWHDp8OjbzogY29ycmVsYcOnw6NvIHBvc2l0aXZhIG91IG5lZ2F0aXZhLCBxdWFuZG8gw6kgcGVyY2VwdMOtdmVsLiAKICA0LiAqUG9udG9zIGV4dHJlbW9zKiBmb3JhIGRhIGFzc29jaWHDp8Ojby4KICAKUGFyYSByZWxhY2lvbmFyIHZhcmnDoXZlaXMsIHBvZGVtb3MgdXNhciB0w6ljbmljYXMgZGUgdmlzdWFsaXphw6fDo28gY29tbyBncsOhZmljb3MgZGUgZGlzcGVyc8OjbyAoKnNjYXR0ZXIgcGxvdHMqKS4gT3V0cmEgdMOpY25pY2Egw6kgYSAqKmFuw6FsaXNlIGRlIGNvcnJlbGHDp8OjbyoqLgoKCiMjIENvcnJlbGHDp8OjbwoKQ29ycmVsYcOnw6NvIMOpIHVtYSBtZWRpZGEgcXVlIGluZGljYSBzZSBhIHJlbGHDp8OjbyBlbnRyZSAyIHZhcmnDoXZlaXMgw6kgZXN0YXRpc3RpY2FtZW50ZSBzaWduaWZpY2F0aXZhLgoKTyBjb2VmaWNpZW50ZSBkZSBjb3JyZWxhw6fDo28gZGUgKipQZWFyc29uKiogw6kgbyBtYWlzIHBvcHVsYXIgZSB1c2FkbyBwcmluY2lwYWxtZW50ZSBwYXJhIHJlbGHDp8O1ZXMgKmxpbmVhcmVzKi4gT3V0cm9zIGNvZWZpY2llbnRlcyBkZSBjb3JyZWxhw6fDo28gcm9idXN0b3MgZSDDunRlaXMgcGFyYSBvdXRyb3MgdGlwb3MgZGUgcmVsYcOnw6NvIHPDo28gb3MgY29lZmljaWVudGVzIGRlICoqU3BlYXJtYW4qKiBlICoqS2VuZGFsbCoqLgoKCiMjIyBDb2VmaWNpZW50ZSBkZSBDb3JyZWxhw6fDo28gZGUgUGVhcnNvbgoKTyBjb2VmaWNpZW50ZSBkZSBjb3JyZWxhw6fDo28gZGUgKlBlYXJzb24qIGluZGljYSBhIGZvcsOnYSBlIGEgZGlyZcOnw6NvIGRhIGNvcnJlbGHDp8OjbyBsaW5lYXIgZW50cmUgZHVhcyB2YXJpw6F2ZWlzLiBTZXUgY29lZmljaWVudGUgYW1vc3RyYWwgcGFyYSBhcyB2YXJpw6F2ZWlzICp4KiBlICp5KiDDqSBkZWZpbmlkbyBwb3I6CgokJHJfe3h5fSA9IFxmcmFjezF9e24tMX0gXHN1bV97aT0xfV57bn0gXGxlZnQoIFxmcmFje3hfaSAtIFxiYXJ7eH19e3NfeH0gXHJpZ2h0KSBcbGVmdCggXGZyYWN7eV9pIC0gXGJhcnt5fX17c195fSBccmlnaHQpICAkJAoKCgpPIHZhbG9yIGRvIGNvZWZpY2llbnRlICRyX3t4eX0kIHZhcmlhIGRlIC0xIGEgMToKCi0gJCsxJDogUmVsYcOnw6NvIHBvc2l0aXZhIHBlcmZlaXRhOiBxdWFudG8gKngqIGF1bWVudGEsICp5KiBhdW1lbnRhIGxpbmVhcm1lbnRlLgotICQtMSQ6IFJlbGHDp8OjbyBuZWdhdGl2YSBwZXJmZWl0YTogcXVhbnRvICp4KiBhdW1lbnRhLCAqeSogZGltaW51aSBsaW5lYXJtZW50ZS4KLSAkMCQgOiBOw6NvIGjDoSByZWxhw6fDo28gZW50cmUgKngqIGUgKnkqCgpUaXBpY2FtZW50ZSwgc2UgYWRvdGEgZXNzZXMgdmFsb3JlcyBwYXJhIGluZGljYXIgYSBmb3LDp2EgZGEgY29ycmVsYcOnw6NvOgoKLSBTZSAkcl97eHl9IFxndCAwLDckIG91ICRyX3t4eX0gXGx0IC0wLDckOiAqKmNvcnJlbGHDp8OjbyBmb3J0ZSoqCi0gU2UgJDAsMyBcbHQgcl97eHl9IFxsdCAwLDckIG91ICQtMCw3IFxsdCByX3t4eX0gXGx0IC0gMCwzJDogKipjb3JyZWxhw6fDo28gbW9kZXJhZGEqKgotIFNlICQtMCwzIFxsdCByX3t4eX0gXGx0IDAsMyQ6ICoqY29ycmVsYcOnw6NvIGZyYWNhKioKLSBTZSAkcl97eHl9IFxhcHByb3ggMCQ6ICoqbsOjbyBow6EgY29ycmVsYcOnw6NvKiogKG91IG7Do28gc2UgcG9kZSBjb25jbHVpciBuYWRhKQoKCiMjIyMgRXhlbXBsbzogYW7DoWxpc2UgZGUgY29ycmVsYcOnw6NvIGRlIFBlYXJzb24KCkNvbnNlbGhvIGltcG9ydGFudGU6IG7Do28gaMOhIHVtIG7Dum1lcm8gYXBlbmFzIHF1ZSBwb3NzYSBsaGUgcmVzcG9uZGVyIHR1ZG8gc29icmUgYSBhc3NvY2lhw6fDo28gZW50cmUgYXMgZHVhcyB2YXJpw6F2ZWlzLiBBcyBtZWRpZGFzIGRlIGNvcnJlbGHDp8OjbyBxdWUgdm9jw6ogdXNhcsOhIHNlcnZpcsOjbyBtYWlzIHBhcmEgY29tcGxlbWVudGFyIGUgcXVhbnRpZmljYXIgb2JzZXJ2YcOnw7VlcyBmZWl0YXMgZW0gZ3LDoWZpY29zIGRlIGRpc3BlcnPDo28gZG8gcXVlIHBhcmEgbGhlIGd1aWFyLiAKClVtIGV4ZW1wbG8gY2xhc3NpY28gY29tIHF1YXRybyBwYXJlcyBkZSB2YXJpw6F2ZWlzIGN1amFzIGFzc29jaWHDp8O1ZXMgaW50ZXJlc3NhbTogeDEgZSB5MSwgeDIgZSB5MiwgZXRjLiwgbyBbcXVhcnRldG8gZGUgQW5zY29tYmVdKGh0dHBzOi8vZW4ud2lraXBlZGlhLm9yZy93aWtpL0Fuc2NvbWJlJTI3c19xdWFydGV0KToKCmBgYHtyfQpzdW1tYXJ5KGFuc2NvbWJlKQoKIyB1bSBwb3VjbyBkZSByZWFycnVtYcOnw6NvIHByaW1laXJvLiAKIyAocGFyYSBlbnRlbmRlciBtZWxob3IsIHZvY8OqIHBvZGUgcHJvY3VyYXIgc29icmUgdGlkeSBkYXRhIGVtIFIpCnQxIDwtIG1lbHQoc2VsZWN0KGFuc2NvbWJlLCAxOjQpLCBpZCA9IGMoKSkKdDIgPC0gbWVsdChzZWxlY3QoYW5zY29tYmUsIDU6OCksIGlkID0gYygpKQoKZGFkb3MgPC0gZGF0YS5mcmFtZSh2YXJzID0gcGFzdGUodDEkdmFyaWFibGUsICJlIiwgdDIkdmFyaWFibGUpLCAKICAgICAgICAgICAgICAgICAgICB2MSA9IHQxJHZhbHVlLCAKICAgICAgICAgICAgICAgICAgICB2MiA9IHQyJHZhbHVlKQpgYGAKCkNhbGN1bGFuZG8gYSBjb3JyZWxhw6fDo28gbGluZWFyOiAKCmBgYHtyfQpkYWRvcyAlPiUgCiAgZ3JvdXBfYnkodmFycykgJT4lIAogIHN1bW1hcmlzZShjb3JyZWxhY2FvX3BlYXJzb24gPSBjb3IodjEsIHYyLCBtZXRob2QgPSAicGVhcnNvbiIpKQpgYGAKCkUgc2Ugb2xoYXJtb3Mgb3MgZGFkb3M/CgpgYGB7cn0KZ2dwbG90KGRhZG9zLCBhZXModjEsIHYyKSkgKyAKICBnZW9tX3BvaW50KGNvbG9yID0gImRhcmtvcmFuZ2UiLCBzaXplID0gNCwgYWxwaGEgPSAwLjcpICsgCiAgdGhlbWVfYncoKSArIAogIHNjYWxlX3hfY29udGludW91cyhicmVha3MgPSBzZXEoMCwgMjAsIDIpKSArIAogIHNjYWxlX3lfY29udGludW91cyhicmVha3MgPSBzZXEoMCwgMTIsIDIpKSArIAogIGV4cGFuZF9saW1pdHMoeCA9IDAsIHkgPSAwKSArIAogIGZhY2V0X3dyYXAofiB2YXJzKQpgYGAKClF1YXRybyByZWxhw6fDtWVzIGRpZmVyZW50ZXMsIG1lc21hIHF1YW50aWZpY2HDp8Ojby4gUGFyYSBvIHNlZ3VuZG8gZ3J1cG8sIG7Do28gaMOhIHVtYSByZWxhw6fDo28gbGluZWFyLiBObyAzbywgaMOhIHVtYSByZWxhw6fDo28gcGVyZmVpdGEgZW50cmUgYSBtYWlvcmlhIGRhcyBvYnNlcnZhw6fDtWVzLCBjb20gdW1hIGV4Y2XDp8Ojby4gTm8gNG8gZ3J1cG8gbsOjbyBow6EgcmVsYcOnw6NvOyBow6EgdW1hIGV4Y2XDp8OjbyBxdWUgZmF6IHBhcmVjZXIgcXVlIGjDoSB1bWEgcmVsYcOnw6NvLgoKTyBxdWUgb3Mgb3V0cm9zIGNvZWZpY2llbnRlcyBwb2RlbSBub3MgZGl6ZXI/IAoKYGBge3J9CmRhZG9zICU+JSAKICBncm91cF9ieSh2YXJzKSAlPiUgCiAgc3VtbWFyaXNlKHBlYXJzb24gPSBjb3IodjEsIHYyLCBtZXRob2QgPSAicGVhcnNvbiIpLCAKICAgICAgICAgICAgc3BlYXJtYW4gPSBjb3IodjEsIHYyLCBtZXRob2QgPSAic3BlYXJtYW4iKSwKICAgICAgICAgICAga2VuZGFsbCA9IGNvcih2MSwgdjIsIG1ldGhvZCA9ICJrZW5kYWxsIikpCmBgYAoKLS0tLS0tLS0tLS0tLQoKIyNGb3LDp2EgZSB0YW1iw6ltIGRpcmXDp8OjbwoKRGl0byBpc3NvLCBlc3NhIGZpZ3VyYSBhanVkYSBhIGxlbWJyYXIgYSByZWxhw6fDo28gZW50cmUgbyB2YWxvciBlc3BlcmFkbyBkbyBjb2VmaWNpZW50ZSAobGluZWFyKSBlIHbDoXJpb3MgdGlwb3MgZGUgYXNzb2NpYcOnw6NvIGVudHJlIGR1YXMgdmFyacOhdmVpczoKCiFbZGEgd2lraXBlZGlhXShodHRwczovL3VwbG9hZC53aWtpbWVkaWEub3JnL3dpa2lwZWRpYS9jb21tb25zL2QvZDQvQ29ycmVsYXRpb25fZXhhbXBsZXMyLnN2ZykKCgotLS0tLS0tLS0tLS0tCgojIEFsZ3VucyB0ZXN0ZXMgZ2VyYW5kbyBhIHJlbGHDp8OjbwoKUmVsYcOnw6NvIGxpbmVhcjoKCmBgYHtyfQpzZXQuc2VlZCgxMjMpCnggPC0gcm5vcm0oMTAwKSAqIDEwMAp0YW1hbmhvX2RvX2Vycm8gPC0gNTAKeSA8LSAwLjUgKiB4ICsgcm5vcm0oMTAwKSAqIHRhbWFuaG9fZG9fZXJybyArIDIwCgpkZiA8LSBkYXRhLmZyYW1lKHggPSB4LCAKICAgICAgICAgICAgICAgICB5ID0geSkKCmdncGxvdChkZiwgYWVzKHggPSB4LCB5ID0geSkpICsgCiAgZ2VvbV9wb2ludChjb2xvdXIgPSAiZGFya29yYW5nZSIsIHNpemUgPSA0KSArIAogIHRoZW1lX2J3KCkKCmNvcihkZiR4LCBkZiR5LCBtZXRob2QgPSAicGVhcnNvbiIpCgojIERvYnJvIGRlIGVycm8gZW0gdG9ybm8gZGUgdW1hIGZ1bsOnw6NvIGxpbmVhciBmKHgpCmRmJHkgPC0gMC41ICogeCArIHJub3JtKDEwMCkgKiB0YW1hbmhvX2RvX2Vycm8gKiAyICsgMjAKCmdncGxvdChkZiwgYWVzKHggPSB4LCB5ID0geSkpICsgCiAgZ2VvbV9wb2ludChjb2xvdXIgPSAiZGFya29yYW5nZSIsIHNpemUgPSA0KSArIAogIHRoZW1lX2J3KCkKY29yKGRmJHgsIGRmJHksIG1ldGhvZCA9ICJwZWFyc29uIikKYGBgCgpSZWxhw6fDo28gbsOjbyBsaW5lYXIKCmBgYHtyfQpuID0gMTAwCmRmIDwtIGRhdGEuZnJhbWUoeCA9IHJ1bmlmKG4sIG1pbj0gMSwgbWF4ID0gMjApKQpkZiR5ID0gMTAwICogZXhwKC0xLjIgKiBkZiR4KSAjKyBybm9ybShuLCBtZWFuID0gMC4wNSwgc2QgPSAxKQoKZ2dwbG90KGRmLCBhZXMoeCA9IHgsIHkgPSB5KSkgKyAKICBnZW9tX3BvaW50KGNvbG91ciA9ICJkYXJrb3JhbmdlIiwgc2l6ZSA9IDQpICsKICB0aGVtZV9idygpCiAgCmNvcihkZiR4LCBkZiR5KQpjb3IoZGYkeCwgZGYkeSwgbWV0aG9kID0gInNwZWFybWFuIikKY29yKGRmJHgsIGRmJHksIG1ldGhvZCA9ICJrZW5kYWxsIikKCiMgRW0gZXNjYWxhIGRlIGxvZwpnZ3Bsb3QoZGYsIGFlcyh4ID0geCwgeSA9IHkpKSArIAogIGdlb21fcG9pbnQoY29sb3VyID0gImRhcmtvcmFuZ2UiLCBzaXplID0gNCkgKwogIHNjYWxlX3lfbG9nMTAoKSArCiAgdGhlbWVfYncoKQoKYGBgCgotLS0tLS0KCgojIyBDb3JyZWxhw6fDo28gdnMuIENhdXNhbGlkYWRlCgpBVEVOw4fDg08hIENvcnJlbGHDp8OjbyBuw6NvIGltcGxpY2EgY2F1c2FsaWRhZGUuIEFsZ3VucyBtYXRlcmlhaXMgc29icmUgbyB0ZW1hOgoKLSBbRXhlbXBsb3MgZXNkcsO6eHVsb3NdKGh0dHA6Ly93d3cudHlsZXJ2aWdlbi5jb20vc3B1cmlvdXMtY29ycmVsYXRpb25zKSBkZSBmb3J0ZSBjb3JyZWxhw6fDo28sIG1hcyBxdWUgYSBjYXVzYWxpZGFkZSBuw6NvIGZheiBzZW50aWRvLgotIFtBcnRpZ28gbmEgV2lraXBlZGlhXShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS9Db3JyZWxhdGlvbl9kb2VzX25vdF9pbXBseV9jYXVzYXRpb24pIGRpc2N1dGluZG8gbyBhc3N1bnRvLgotIFtBdWxhIGludGVyZXNzYW50ZV0oaHR0cHM6Ly93d3cua2hhbmFjYWRlbXkub3JnL21hdGgvcHJvYmFiaWxpdHkvc2NhdHRlcnBsb3RzLWExL2NyZWF0aW5nLWludGVycHJldGluZy1zY2F0dGVycGxvdHMvdi9jb3JyZWxhdGlvbi1hbmQtY2F1c2FsaXR5KSBzb2JyZSBvIHRlbWEuCgoKIyMgQ3VpZGFkbyBjb20gc3Vtw6FyaW9zCgpgYGB7cn0KZGF0YXNhdXJ1c19kb3plbiAlPiUgCiAgZ3JvdXBfYnkoZGF0YXNldCkgJT4lIAogIHN1bW1hcmlzZV9hbGwoZnVucyhtZWFuLCBzZCkpCmBgYAoKIyMgQ29ycmVsYcOnw7VlcwoKYGBge3J9CgpkYXRhc2F1cnVzX2RvemVuICU+JSAKICBncm91cF9ieShkYXRhc2V0KSAlPiUgCiAgc3VtbWFyaXNlKGNvcnJlbGFjYW8gPSBjb3IoeCwgeSkpCgpgYGAKCgojIyBCb3hwbG90cwoKYGBge3J9CmRhdGFzYXVydXNfZG96ZW4gJT4lIAogIGdyb3VwX2J5KGRhdGFzZXQpICU+JSAKICBnZ3Bsb3QoYWVzKHggPSBkYXRhc2V0LCB5ID0geSkpICsgCiAgZ2VvbV9ib3hwbG90KCkKYGBgCgpgYGB7cn0KZGF0YXNhdXJ1c19kb3plbiAlPiUgCiAgZ3JvdXBfYnkoZGF0YXNldCkgJT4lIAogIGdncGxvdChhZXMoeCA9IGRhdGFzZXQsIHkgPSB5KSkgKyAKICBnZW9tX3Zpb2xpbigpCmBgYAoKCmBgYHtyfQpnZ3Bsb3QoZGF0YXNhdXJ1c19kb3plbiwgYWVzKHg9eCwgeT15LCBjb2xvdXI9ZGF0YXNldCkpKwogIGdlb21fcG9pbnQoKSsKICB0aGVtZShsZWdlbmQucG9zaXRpb24gPSAibm9uZSIpKwogIGZhY2V0X3dyYXAofmRhdGFzZXQsIG5jb2w9NCkKCmBgYAoK